用 Lagrange 乘子法求出主成分. 根据 (1,2), 定义 Lagrange 函数$$f_1(\lambda )={\alpha }_1^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_1-\lambda ({\alpha }_1^{\mathrm{T}}{\alpha }_1-1),$$令$$\frac{\partial f_1}{\partial {\alpha }_1}=0\Rightarrow \mathbf{\Sigma }{\alpha }_1=\lambda {\alpha }_1.$$因此${\displaystyle \lambda }$是${\displaystyle \mathbf{\Sigma }}$的特征值, ${\displaystyle {\alpha }_1}$是对应的单位特征向量, 目标函数改写为$${\alpha }_1^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_1={\alpha }_1^{\mathrm{T}}\lambda {\alpha }_1=\lambda .$$因此, 目标函数最大化意味着${\displaystyle \lambda }$最大化, 从而取${\displaystyle {\alpha }_1,{\lambda }_1}$对应于${\displaystyle \mathbf{\Sigma }}$的最大特征值, ${\displaystyle {\alpha }_1^{\mathrm{T}}\mathbf{x}}$构成第一主成分, 且$$\mathrm{Var}({\alpha }_1^{\mathrm{T}}\mathbf{x})={\alpha }_1\mathbf{\Sigma }{\alpha }_1={\lambda }_1.$$
接下来求解第二主成分, 等价于优化问题$$\begin{array}{rl}\underset{{\alpha }_2}{max}& {\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2,\\ \mathrm{s}.\mathrm{t}.& {\alpha }_1^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2={\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_1=0,{\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2=1.\end{array}$$这样$$0={\alpha }_1^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2={\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_1={\alpha }_2^{\mathrm{T}}{\lambda }_1{\alpha }_1={\lambda }_1{\alpha }_2^{\mathrm{T}}{\alpha }_1={\lambda }_1{\alpha }_1^{\mathrm{T}}{\alpha }_2.$$
定义 Lagrange 函数$$f_2(\lambda ,\varphi )={\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2-\lambda ({\alpha }_2^{\mathrm{T}}{\alpha }_2-1)-\varphi {\alpha }_2^{\mathrm{T}}{\alpha }_1.$$令$$\begin{array}{rl}& \frac{\partial f_2}{\partial {\alpha }_2}=2\mathbf{\Sigma }{\alpha }_2-2\lambda {\alpha }_2-\varphi {\alpha }_1=0\\ \Rightarrow & 2{\alpha }_1^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2-2\lambda {\alpha }_1^{\mathrm{T}}{\alpha }_2-\varphi {\alpha }_1^{\mathrm{T}}{\alpha }_1=0-\varphi =0\Rightarrow \varphi =0.\end{array}$$ 于是$$\frac{\partial f_2}{\partial {\alpha }_2}=0\Rightarrow \mathbf{\Sigma }{\alpha }_2=\lambda {\alpha }_2,$$目标函数改写为$${\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2={\alpha }_2^{\mathrm{T}}\lambda {\alpha }_2=\lambda .$$取${\displaystyle \mathbf{\Sigma }}$的第二大特征值${\displaystyle {\lambda }_2}$和对应的单位特征向量${\displaystyle {\alpha }_2}$即可, 从而${\displaystyle {\alpha }_2^{\mathrm{T}}\mathbf{x}}$是第二主成分, 方差为$$\mathrm{Var}({\alpha }_2^{\mathrm{T}}\mathbf{x})={\alpha }_2^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_2={\lambda }_2.$$
以此类推, 得到${\displaystyle k}$个主成分.

2. 根据 这里 以及迹的性质, 我们有$$\begin{array}{rl}\sum _{i=1}^m\mathrm{Var}(x_i)=& \mathrm{tr}({\mathbf{\Sigma }}^{\mathrm{T}})=\mathrm{tr}(A\mathrm{\Lambda }{A}^{\mathrm{T}})=\mathrm{tr}(A^{\mathrm{T}}\mathrm{\Lambda }A)\\ =& \mathrm{tr}(\mathrm{\Lambda })=\sum _{i=1}^m{\lambda }_i=\sum _{i=1}^m\mathrm{Var}(y_i).\end{array}$$
3. 定义${\displaystyle e_i}$为基本单位向量, 则$$\mathrm{Cov}({\alpha }_k^{\mathrm{T}}\mathbf{x},e_i^{\mathrm{T}}\mathbf{x})={\alpha }_k^{\mathrm{T}}\mathrm{Cov}(\mathbf{x},\mathbf{x})e_i={\alpha }_k^{\mathrm{T}}\mathbf{\Sigma }{e}_i=e_i^{\mathrm{T}}\mathbf{\Sigma }{\alpha }_k=e_i^{\mathrm{T}}{\lambda }_k{\alpha }_k={\lambda }_k{\alpha }_{ik}.$$从而$$\rho (y_k,x_i)=\frac{\mathrm{Cov}(y_k,x_i)}{\sqrt{\mathrm{Var}(y_k)\mathrm{Var}(x_i)}}=\frac{\mathrm{Cov}({\alpha }_k^{\mathrm{T}}\mathbf{x},e_i^{\mathrm{T}}\mathbf{x})}{\sqrt{{\lambda }_k}\sqrt{{\sigma }_{ii}}}=\frac{\sqrt{{\lambda }_k}{\alpha }_{ik}}{\sqrt{{\sigma }_{ii}}}.$$
4. 由 3,$$\sum _{i=1}^m{\sigma }_{ii}{\rho }^2(y_k,x_i)=\sum _{i=1}^m{\lambda }_k{\alpha }_{ik}^2={\lambda }_k{\alpha }_k^{\mathrm{T}}{\alpha }_k={\lambda }_k.$$
5. 一方面, ${\displaystyle y_1,\cdots ,y_m}$互不相关, 故$${\rho }^2(x_i,(y_1,\cdots ,y_m))=\sum _{k=1}^m{\rho }^2(y_k,x_i).$$另一方面, ${\displaystyle x_i}$可以表示为${\displaystyle y_1,\cdots ,y_m}$的线性组合, 故$${\rho }^2(x_i,(y_1,\cdots ,y_m))=1.$$